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ABSTBACT 

A nethpd of/individual instructor classroom research 
is proposed and deaonstrat^id in this paper. By capitalizing on 
baseline data collected Jron a similar, prior class and foo^sing 
treattifent on difficul't^est items, the instructor can develop 
sensitive Within cla^ designs for the detection of treatment 
effects « An experim^t using written objectives given to th^ students 
to aid their lea^^ng and using the single classroom design shovs 
that bbjective^^ignif icantly imprpve learning. Written objectives 
are also s\^eil:n to interact with item ' difficulty . The experiment leads 
to the r^erfcomm^ndation that instructors engage in personal in^,4JFX,^ 
into tMir efforts to influence student learning and that such ..Cr,^ 
inquiry be added to the educational research literature. (Authairi'''*'* 
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Abstr 




A method of Indlvl^u^ Instructor classroom research Is proposed 
and demon^trAted In -th^s paper. capitalizing on baseline data 
collected from a similar, prior class and focussing treatment upon \ 
difficult test Items, the Instructor can develop sensitive within 
class designs lEor the detection of treatment effects. An experiment 
using written objectives given to the students' to aid their learning 
and using the single daodroom design shows that objectives significantly 
Improve learning. Written objectives are also ahown to Interact with 
item dlfflculty.\ The experiment leads to the recommendation that ln«* 
structors engage \ in personal Inquiry into their efforts to influence • 
student learning \and that such inquiry be added to the educational 
research literatuire. 
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Pe^Bonal Inquiry In the Classroom: An Alternative Approach to 

Educational Research - 

Inferential statistics has been developed to solve, In a quantitative 
vimner > the problems Inherent In generalizing from a specific sample to an 
abstract populatloti. Even with the cautions provided by Inferential 
statiatlic^^a common errot; mad^ In their use Is over generalization. This error 
suggests that, we hold the goal or value of explaining the worW in general tod 
dearly and consequently bias Qur thinking in the direction of too much absti^c-* 
tion. In educational research tdtms this translates Into trying to find 
instructional methods appropriate in all classrooms for all instructors. 
Perhaps this dream is tqo grandl^ose. The JLndlvidual Jteacher is primarily con- 
cerned with his class^ his studei^ts, and his ability to Influence those students 
in ways he^ judges to be favorable. The teacher is engaged in the exercise of 
personal Influence and neeids a method of inquiry that will allow him to assess 
the> oiiitcome "bf his particular infli/ence attempts. The only glenerallzation he 
needs to be concerned with is that of transferring his Impact to a new group 
of studetitai. 

Educational research needs to mo^e to the^J^dividual instructor's claks*^^^ 
room if rapid progress on theories of instruction is to be made^possible. There 
are two. reasons for such' a move, (a) The cost of doing xesearctr^wtth many 
classrooms, several teachers ^ and large numbers of students is too great; we 
will never have a large number of such studies I This approach to research la . 
surely needed; but because of its limits, effort and opportunity, cannot be the 
major source of educational data* (b) The short laboratory experiment on the 
othjer hand should not serve as the major data base for Instructional theory 
because of the possible distoiftions introduced into the data due to the small 
size and simplicity of the laboratory situation. For example, a subject asked 
to learn short passages of prose In a one hour periqd^f ollowed by Immediate 
testing may not engage in th^ same behaviors he would use in reading a textbook 
over a period of several weeks. While the laboratory tan certainly suggest 
potentially relevant variables to the individual instructor, the expectation of 
generalization to the classroom, while positive, must remain low until extensive 
classroom tests have been made. However, the individual classroom*' has the right 
size and complexity from the viewpoint of the learning process and is the right 
size from the viewpoint of convenience in the research effort. 
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If the individual Instructor Is to be the principal researcher, he needs 
a sharper set of research methods than Is normally provided. The stability of 
item difficulties from quarter to quarter and class to class opens the possl- 
blllty of Improved research techniques (Anderson, 1975). « In the Anderson 
study, Item difficulty Is defined as the proportion of students choosing the 
correct response to a test Item^^ Item difficulties werc^ calculated for 
Identical examinations given to students In several sections of a Fortran 
programming class and these Item difficulties were then correlated on an Item 
by Item basis. The average Intercorrelatlon of flv^ classes In the ftutumn 
of 1974 was .73 and the average Intercorrelatlon of eight classes in the winter 
of 1975 was .71. These high correlations were found In spite of considerable 
variation In the sections, e.g., different Instructors, different practice 
problems, different lectures, etc. These findings Indicate tliiat material 
easily^ learned by one class will be easily learned by another class and that 
difficult material In one cla^s will prove difficult; In a second class. Be-' 
cause of the commonality demonstrated by the high correlations between classes ^ 
we would expect data from oiie quarter to be useful as a reliable baseline for 
data gathered in subsequent quarters. The high correlations encourage the 
suBtraqtion of baseline item difficulties from item dlfflcultlea obtained 
during tK^ treatment quarter in order to arrive at a change score which more 
sensitively reflects the effect of the treatment. 

The effect of dianging classes (different subjects) can' be ruled out by 
applying the ti;^atmeiit to part of the tx>-be-learned material. This step 
allows a Vlthin class comparison of the change from baseline for untreated 
fltnd treated^ item difficulties. 

One final touch is needed bec|iuse the Item difficulties for a test given 
in a 14-ve clati^room tend to form a skewed distribution with the peak above the 
mean difficulty* Items which baselijne at high difficulty values (easy items) 
do not have much room for change, i.e. there may i6e a ceiling effect on the 
majority of the test items. The group of items selected for treatment should 
thus consist of one'-half of the items having low difficulties in the baseline 
period* Treatment is confined to one-^half of the difficult items because the 
remaining difficult itenfs are needed as a control for regression effects. 
Baselining item difficulties, within.class design, and focus of treatment 
analysis on difficult items should all contribute to ' the sensitivity of the 



classroom design as a research tool. (These design features strongly resemble 
the research methods developed by the behavior modifiers, Bandura, 1969, and 
their work should provide many useful cues to the classroom Insttructor engaged 
In Inquiry Into his personal effectiveness.) 

The remainder of this paper Illustrates the classroom design features 
Just mentioned In connection with a specific treatment. Duchastel and Merrill 
(1973) reviewed several classroom and laboratory studies which assessed the 
effectiveness of written objectives In Improving student performance. Approx- 
lmat,ely 45Z of these studies showed objectives to be beneficial. However, 
many of the comparisons made In the classroom were poorly controlled. I.e., 
different classes were used for control and treatment. Item difficulties were 
not used, and there was no focus on difficult Items. The purpose of this study 
Is thus two fold: (a) to provide a demonstration of an Individual classroom 
research methodology and (b) to determine whether or not presenting students 
with written objectives will favorably Influence their l^arnipg. 
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Methods ■ , 

Subjects . Baseline Item difficulties were colljected from 64 Introductory 
Psychology atud^nts at Shoreline Community College la Seattle, Washington. 
There were twq separate- Ibut similar sections of the introductory class; each 
Section was given the same tests and the data from tie two sections was pooled 
for scotlng. The experiment proper was conducted wi:h 54 students in an evening 
Introductory Psychology class at the ynlverslty of WjishlngtOn. 

Course Materials . The reading assignments for both courses were taken 
from Beach, Psychology : Core Concepts and Special Toj>ics, 1973, and from a 
book of readings provided with the test J^^jursen & B^ach, Readings in Psychology : 
Core Concepts and Special Topics , 1973). ^.The page nymbers of the reading assign- 
ments were given to the students at least Wo weeks ]>r lor to the testing dates. 
The rest of the content of the course was presented via lectures. An effort 
.was made to prevent overlap in the content of the readings and lectures and 
the author estimates that the effort was 85% successi'ul. 

Research design . During the first four weeks ol' 
(U of W), the students were given, at tha beginning oi 
graphed sheet which had statements of the major *point:c 
as objectives. For example. 



the experimental quarter 
f each lecture, a mlmeo-- 
s of the lecture written 



Each student will recognize an unobtrusive measure as 
a measurement which does not alter real world events during 
its' use in measuring. 

Given a new experiment testing Barker *s ecological 
theory and the knowledge that the. experiment confirmed the 
theory^ each student will recognize the outcome of the 
experiment. 

The students were given 63 such objectives during the four week period. No 
objectives were provided for the reading assigiuaents during this period. In 
the second four weeks of the quarter a reversal occurred; 50 objectives were 
given with the reading assignments and no ^objectives were givcp with lectures. 
There was no experimentation or data analysis during the last two weeks of the 
quarter or during the week of final examinations. The objectives were written 
af teir the text and lectures were XTritten. and so they se^ed primarily to make 
explicit the goals for learning Implicitly embodied in tlie written material. 

St^^dent performance was measured with multiple choice test questions. 
The questions which corresponded to the example objectives are as follows: 

A museum director measures the wear and tear on the 
floor tiles In front of pictures to assess their popularity. 

This is an example of .. . 

A. an unobtrusive measure.. 

B. operator^ generated data* 

C. a discriminative stimuli. 

D. an ex|>eriment^l manipulation. 

Wicker tested Barker's conclusions conqeming the under- 
manning and ov^rmfilnnlng of behavior settings in small (338 
member) and large (1559 member) churches. The churches were 
both in the same large city, were the eame denomination^ and 
had members from the same socioeconomic class. His findings 
agreed with Barker's population studies. Which of the follow- 
ing is, true of Wicker's findings? 

A. The larg^^^hurch offered a greater variety of 
behavior setting. 

B. The members the small church participated in a 
greater variety of behavior setti.ngs. 

C. Small church/members donated more money per year. 

D. All of th^above. 

The responses of the experimental University of Washington students were turned 
into an item difficulty (proportion of Cftudents responding correctly) from which 
the item difficulty for the Shoreline students was subtracted. The resulting 
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change tcores were separated on two dmenslons, (a)- presence or absence of 
objectives and (b) high (above 60%) cfc low (below 59%) Item difficulty, and 
analyzed with analysis of variance tfechnlques. The hl^h or low difficulty 
split was done using the baseline difficulties collected at Shoreline 
Community Colliage. 

Results 

There were 37 test Items i.n common between the Shoreline and University 
of Washington classes during /the first four week period,. 22 of those related 
to the text and 15 related /o the lectures. The 22 Items related to the text 
had a Pearson product mome/t correlation iof .83 between the two classes whereas 
the 15 Items related to the lectures (objectives present) correlated .52. For 
the second four week period, there were 40 common Items, 22 related to the 
text (objectives present) and 18 to the lectures. In this period the lecture 
question Item difficulties correlated ^79 )and the text question Item diffi- 
culties correlated .52. Note that whei?e nd changes were Introduced to the 

■ ■ ■ fl, 

course content the correlation of the Shoreline baseline difficulties with the 
University of Washington difficulties high (.83 and .79) whereas change ef- 
forts lowered the correlations obtalne|* (.52 and .52). Considerable commonality 
exists In the performance oi these twp classes. 



More to the polnt^are the reSultji 



pf a two X two unweighted means analysis 



of variance (Myers, 1972, p. 116) performed on the difference scores. Recall 
that each difference score was obtained by subtracting the Item difficulty of 
the 1^^ Item from the Shoreline class from the Item difficulty of 1^^ Item at 
the University of Washington. Table I shows the mean change scores for each 
cell while table 2 reports the analysis of variance data* The presence of 
objectives significantly Improves student performance, the change ^cores are 
Qlearly larger with moxe difficult Items, and the significant objectives X 
difficulty Interaction shows that the objectives had their biggest impact on 
the dlffldult Items. These effects are discoverable with the design used even 
though the University of Washington students performed In general at a higher 
level. 
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Table 1 



Objectives 
present 

No objectives 



Change Score Means 










'.I 


Proportion Correct at Shoreline 




• 0 - .59 


.60 - 1.00 




.29 


.7 




* fa ■ 
N - 19 


N - 


18 


.12 


.5 




N - 14 


- N - 


26- 



N is the number of test question change scores 
Included in the cell. ^ 

/ 
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Table 2 







Analysis of Variance Data 
• -- 




SV 

9 


* 

df 


SS 


MS 


F 






.009 






Objectives 


1 


.009 


11.39 










** 


Difficulty 


1 


.021 


.021 


26.58 


Obj X Dlf f 


1 


.0056 


.0056 


* 

7.09 


S/Obj X Dlff (AdJ) 


73 


.0578 


.00079 





01 



p < .001 



Discussion 



Tht methods of experimental analysis used In this study showed the presence 
of objectives to make a significant difference in the performance of introductdry 
psychology students (p < .001) « It is interesting to note that without the 
sensitivity of the analysis performed these differences would not hkve been found. 
An independent t test comparing tKe item difficulties of items with objectives 
With the item difficulties 6f the itisms without objectives showed no significant 
difference between the two types of items (t^^ » 1.02, p > .10). The finer 
analysis allowed by using baseline data from previous classes is clearly needed 
jlf we are to assess the effect of single treatment variables in complex class* 
room situations. 

The interaction of item difficulty with the written-objectives tfreatment 
points to a common weakness in the design of classroom experiments. We need to 
focus our analysis on the more sensitive test items if we are to detect the full 
effect of our treatment. Ceiling effects work against the detection of differ- 
ences between the treatment and a control condition and lead us to conclude 
that our treatment is not effective when in fact it may be quite effective. 
This interaction clearly points to the need for Instructional improvement 
efforts which focus on selected segments of course content, and so we are led 
to the same focus on difficult material in the classroom that paired-associate 
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tftaearchers (Atkinson, 1972; AtlclnBoa & Paulson, 1972) have used in laboratory 

Studies/ ' ' 

There are .limitations to this type of study. Will the results generalize? 
It Is possible that the author has a talent for writing objective^ which other 
Instructor's do not sha'te, but that floes not seem likely. The objectives are 
nothing more thaii a statement of the key concepts the students are to learn. 
Perhaps the effect of objectives discovered here Is partly due to the nature 
of Introductory psychology or the multiple choice testing format. Until this 
experiment Is conducted In other classrooms, with different content and 
different testing methods, the question, "Will the results generalize?", caoMt 
be rigorously answered.. Certainly the results of this one experiment should 
lead-to the positive expectation that objectives (or any other variable produc- 
ing positive results In the classroom) will make a dlffereflce elsewhere. A 
second problem concerns the Inability of the single classroom design to Introduce 
variability on dimensions which may be relevant to student performance. For 
example, the individual Instructor would normally be seen as, having a single, 
unvarlable constellattDn of personality traits. Research with variables like 
personality will have to Involve more than one instructor. 

All activities 'engaged in- by humans are at some fundamental level personal. 
The individual Instructor teaches his students, his course and this personal 
environment is where the Instructor must strive to Improve his ability to teach. 
If he wants to be systematic about such Improvement efforts, he needs research 
designs which allow personal inquiry, feeneralization to other Instructor's 
classes, while it may occur, is not the issue. In making the Individual class- 
room the location of emphasis in the educational research effort, we are 
focussing on thc^one place where research can find a vigorous home. The number 
of opportunities for such researdh is large and the effort needed is often no 
more than the effort teachers normally expend to improve their courses. Eventually 
enough single case experiments such as the experiment reported here will accumulate 
so that generalizations may appear. Even if the research outcomes prove situa- 
tion specific, the methods used to discover the particular results will generalize. 
The Instructor need only adapt the research methods to his personal Inquiry. 



/ 
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